The present invention is related to processes and apparatus for transferring information between computers, particularly between a client computer and a server computer where the information is stored.
In most computer networks it is desirable to have the capability to download, i.e., transfer, data from one computer to another. Typically, data is downloaded from one computer on the network, such as an information provider""s site on the Internet, to another site, i.e., computer, where the data is to be used. A file containing data, such as an executable program, graphics or other information, typically is made available for download at one or more sites. The availability of the file is advertised to potential users. Individuals who are interested in using the file access the site to download the file. This kind of information distribution reduces costs and enables efficient tracking of the use of the information.
There are several applications that provide a protocol for downloading files from a server computer to a client computer on a network. Example applications which use the Internet or other TCP/IP-based network include servers and clients that implement the hypertext transfer protocol (HTTP) and the file transfer protocol (FTP). A particular problem with downloading information using applications that support these protocols is that the server application relies solely on the underlying transport protocol for reliability in the delivery of the data. If an error occurs during transmission of the data, the download simply terminates. For the download to complete successfully, the operation must be manually tried again, and the entire file must be downloaded. Such a process can be time consuming and frustrating, especially if the download is almost complete when a failure occurs.
However, the FTP specification, defined in Internet Request For Comments (RFC) 959, includes a restart procedure by which an interrupted FTP service command can be restarted from the point where it was interrupted. This restart procedure is defined for only two of the three modes in which data transfer can occur: block mode and compressed mode. In block mode, data is transmitted as a series of data blocks preceded by one or more header bytes. One of these header bytes includes descriptor codes, which may indicate a restart marker. In compressed mode, transmitted data includes regular data, compressed data and an escape sequence of two bytes. The escape sequence also includes descriptor codes that have the same meaning as in block mode.
To support restart in FTP, the sender of data must send data in block mode or compressed mode and insert a restart marker, or marker code, in the data stream with some marker information. The marker information has,meaning only to the sender, and could represent a bit-count, a record-count, or any other information by which a system may identify a data checkpoint. The receiver of data, if it implements the restart procedure, then marks the corresponding position of this marker in the receiving system. In the event of a failure, the user sends a command called RESTART, with a marker code as its argument. The sender then skips over the file specified by the marker code to the data checkpoint specified by the marker code. The RESTART command must be immediately followed by whatever service command was interrupted, such as a read (RETR), write (STOR), directory (LIST) or append (APP). This restart procedure requires both the server to maintain a mapping between data checkpoints and marker codes for each operation and the client to monitor the marker codes received. Moreover, these commands are initiated manually by a user using the FTP client application.
Most currently available server and client programs that support FTP do not support block or compressed mode transfers, and generally support only a third mode of transfer, called stream mode. Most browsers for the Internet also use only this mode of transfer for communication using HTTP. In stream mode data is transmitted as a stream of bytes, without restriction on the representation type used. If the structure of the data is a file structure, an end-of-file (EOF) indication is indicated by the sending host closing the data connection and all bytes are data bytes. The FTP specification does not define any restart procedure for stream model transfers. Accordingly, most currently available server and client programs that support FTP also do not support the RESTART command. If a failure occurs during a download, the operation must be manually tried again, and the entire file must be downloaded, obliging an individual to be present during the download.
Similarly, browsers using the HTTP protocol do not support any restart procedure. A proposed specification for a new version (1.1) of HTTP includes a range header in a GET message to enable partial transfers and is intended to reduce unnecessary network usage. See Internet Request for Comments (RFC) 2068. However, the use of a partial GET command by the client is not specified. The HTTP 1.1 specification as proposed also states that a client should retry a request if a connection closes before any status, or a continue response, is received from the server. However, there is no specification regarding error handling if data is received from the server before a connection closes. Apparently, if a failure occurs during a download, the operation must be manually tried again, and the entire file must be downloaded, obliging an individual to be present during the download.
Accordingly, a general aim of this invention is to provide a download process and mechanism that simplifies the download process, and improves the likelihood of successful completion of the download. This functionality also allows the individual not to be present during the download.
In the present invention, a download is monitored by a client application and is restarted automatically if a failure occurs. Data read during the restarted download is appended to the existing file. Termination of a download might occur due to a failure of the server system or failure of the network or for many other reasons. These failures can be detected, for example, by monitoring whether valid data has been reliably received at the client within a specified period of time. Automatically restarting the download after such failures increases the likelihood of successful completion.
In a particular embodiment, the server application can transfer the data to the client as a stream of data with little or no formatting or processing, for example by using the stream mode in FTP. In this embodiment, the client monitors the amount of data reliably received. In case of a failure, the client automatically sends another request to the server, instructing the server to start reading the file from a specified offset, determined by the amount of data already received. In this embodiment, there is no need for marker codes or other processing to be performed and tracked by the server.
In another embodiment, the client program is specially adapted for performing only read or retrieve requests which reduces the size of its program code. In response to a request to download a file, the client program is downloaded first from the server computer. The location of the requested file may be stored, or hard-coded, within the program. The client program is then executed on the client computer to transfer the requested file.
In another embodiment, the client program is used to download files that are made available through other services, such as by a listing in a document published in the hypertext markup language (HTML) via an HTTP server connected to a network. An HTML browser that accesses and displays the HTML file can display the files available for download as hypertext links. Selection of a hypertext link viewed in the browser causes the client program to be executed to download the requested file. In this embodiment, the client program may be resident at the client computer as a separate program, may be part of the browser or can be downloaded from the server before execution.
In these embodiments, the client program can be executed in response to some action by another program, at a time specified by the user or upon command from the user. By allowing a user to specify a time for a download operation in combination with the ability to automatically restart a download, the user may be absent, yet have an increased likelihood that the download will be successful. The various embodiments can also be used in combination with each other.
Accordingly, one aspect of the present invention is a process or client system for downloading a data file from a server computer to a client computer, wherein the data file is stored on a computer readable medium connected to the server computer. The process and client system have an application protocol for sending data as a stream of data using a transport protocol over a computer network connecting the client computer to the server computer. The client computer initiates a download of the data file by sending, using the application protocol, a request for the data file to the server computer over the computer network. Data received from the server computer as a stream of data in response to the request for the data file is monitored to detect termination of the download of the data file and to track an amount of data from the data file successfully received by the client. The amount of data successfully received by the client is stored when an error is detected that terminates the download. In response to a detected error that terminates the download of the data file, the download of the data file is automatically restarted by initiating a second download of the data file by sending, using the application protocol, a request for the data file to the server computer, wherein the request specifies the amount of data successfully received.
In one embodiment, the download is initiated by sending a message to the server computer using a second application protocol requesting an indication of a port for a server process for performing the download. When indicator of the port is received, a message is sent to the server process using the application protocol at the indicated port to request the data file.
In another embodiment, the download is initiated by receiving from a user a specified time for performing the download of the data file. The message is sent to the server computer at the specified time.
Another aspect of the invention is a process and client system for downloading a data file from a server computer to a client computer, wherein the data file is stored on a computer readable medium connected to the server computer, wherein the process sends data over a computer network connecting the client computer to the server computer. The client sends a request for a client program to the server computer over the computer network. The client program is receiving from the server computer, in response the request. The client program when executed initiates a download of the data file by sending a request for the data file to the server computer over the computer network. The data received from the server computer in response to the request for the data file is monitored to detect termination of the download of the data file. In response to a detected error that terminates the download of the data file, the download of the data file is automatically restarted by sending a request for the data file to the server computer.
In one embodiment, the download is monitored by tracking an amount of data from the data file successfully received by the client. The amount of data successfully received is stored when an error is detected that terminates the download. In this embodiment, the download of the data file may be automatically restarted by initiating a second download of the data file by sending a request for the data file to the server computer, wherein the request specifies the amount of data successfully received. This embodiment may be particularly useful where the application protocol sends data as a stream of data.
In another embodiment, the download is initiated by sending a message to the server computer requesting an indication of a port for a server process for performing the download. After the indicator of the port is received by the client a message is sent to the server process at the indicated port requesting the data file.
In another embodiment, the client program, when executed, receives from a user a specified time for performing the download of the data file. The message is sent to the server computer at the specified time.
Another aspect of the present invention is a process or server system for downloading a data file from a server computer to a client computer, wherein the data file is stored on a computer readable medium connected to the server computer. The process uses an application protocol for sending data as a stream of data using a transport protocol over a computer network connecting the client computer to the server computer. A request is received for the data file from the client computer over the computer network, using the application protocol. Data is readfrom the data file and sent as a stream of data to the client computer using the application protocol. If the transfer terminates, after such a termination a request for the data file is received from the client computer, wherein the request specifies the amount of data successfully received. The data is read from the data file starting from an offset determined according to the specified amount of data and is sent as a stream of data to the client computer using the application protocol.
Other aspects of the invention include a computer readable medium on which executable program logic is stored, which when executed implements the client system or the server system described above. Another aspect of the invention is a combination of the client system and the server system. Yet another aspect of the invention is the combined process performed by the combination of the client system and the server system.
In all aspects of the invention, the transport protocol may be TCP or other protocol. The network protocol may be, for example, the Internet protocol (IP), allowing the invention to be used on the Internet or a private network. The computer network may be an ethernet, an asynchronous transfer mode network, or any other kind of network.
By using a client that monitors a download and automatically restarting the download from the offset at which a failure occurs, the user does not need to be present to make sure a download is successful. By using a server which starts reading and sending data from a data file from a specified offset within the data file, an interrupted download may be restarted where it left off. The combination of this client and server system increase the likelihood of success of the download and eliminate unnecessary data transfer.